[Development of MEDUC-PG14 survey to assess postgraduate teaching in medical specialties]. / Evaluación de docentes clínicos de Postgrado: desarrollo y propiedades psicométricas del instrumento MEDUC-PG14.

BACKGROUND: Feedback is one of the most important tools to improve teaching in medical education. AIM: To develop an instrument to assess the performance of clinical postgraduate teachers in medical specialties. MATERIAL AND METHODS: A qualitative methodology consisting in interviews and focus-groups followed by a quantitative methodology to generate consensus, was employed. After generating the instrument, psychometric tests were performed to assess the construct validity (factor analysis) and reliability (Cronbach’s alpha). RESULTS: Experts in medical education, teachers and residents of a medical school participated in interviews and focus groups. With this information, 26 categories (79 items) were proposed and reduced to 14 items (Likert scale 1-5) by an expert’s Delphi panel, generating the MEDUC-PG14 survey, which was answered by 123 residents from different programs of medical specialties. Construct validity was carried out. Factor analysis showed three domains: Teaching and evaluation, respectful behavior towards patients and health care team, and providing feedback. The global score was 4.46 ± 0.94 (89% of the maximum). One teachers’ strength, as evaluated by their residents was “respectful behavior” with 4.85 ± 0.42 (97% of the maximum). “Providing feedback” obtained 4.09 ± 1.0 points (81.8% of the maximum). MEDUC-PG14 survey had a Cronbach’s alpha coefficient of 0.947. CONCLUSIONS: MEDUC-PG14 survey is a useful and reliable guide for teacher evaluation in medical specialty programs. Also provides feedback to improve educational skills of postgraduate clinical teachers.

Evaluación de docentes clínicos de Postgrado: desarrollo y propiedades psicométricas del instrumento MEDUC-PG14 / Development of MEDUC-PG14 survey to assess postgraduate teaching in medical specialties

Background: Feedback is one of the most important tools to improve teaching in medical education. Aim: To develop an instrument to assess the performance of clinical postgraduate teachers in medical specialties. Material and Methods: A qualitative methodology consisting in interviews and focus-groups followed by a quantitative methodology to generate consensus, was employed. After generating the instrument, psychometric tests were performed to assess the construct validity (factor analysis) and reliability (Cronbach’s alpha). Results: Experts in medical education, teachers and residents of a medical school participated in interviews and focus groups. With this information, 26 categories (79 items) were proposed and reduced to 14 items (Likert scale 1-5) by an expert’s Delphi panel, generating the MEDUC-PG14 survey, which was answered by 123 residents from different programs of medical specialties. Construct validity was carried out. Factor analysis showed three domains: Teaching and evaluation, respectful behavior towards patients and health care team, and providing feedback. The global score was 4.46 ± 0.94 (89% of the maximum). One teachers’ strength, as evaluated by their residents was “respectful behavior” with 4.85 ± 0.42 (97% of the maximum). “Providing feedback” obtained 4.09 ± 1.0 points (81.8% of the maximum). MEDUC-PG14 survey had a Cronbach’s alpha coefficient of 0.947. Conclusions: MEDUC-PG14 survey is a useful and reliable guide for teacher evaluation in medical specialty programs. Also provides feedback to improve educational skills of postgraduate clinical teachers.

Measuring the educational environment in ambulatory settings / Medición del ambiente educacional en contextos ambulatorios

BACKGROUND AND OBJECTIVES: Students' perceptions of their educational environment (EE) have been studied in undergraduate and postgraduate curricula. The Ambulatory Care Learning Educational Environment Measure (ACLEEM) is an inventory that was recently developed to measure the EE in postgraduate ambulatory settings. The aim of this study was to assess the psychometric properties of the inventory. METHODS: A mixed methodology was used to develop the ACLEEM including: Grounded theory (8 focus groups); a two-round Delphi technique to identify consensus; and a pilot study. The inventory was refined to 50-items after the pilot study and it was prospectively administered to a large cohort of clerks and residents in Chile during 2010-2011. Psychometric measurements included factor analysis followed by Varimax rotation for construct validity, Cronbach's alpha coefficients for internal consistency and Generalizability theory for test reliability. RESULTS: Four-hundred and eleven students responded: 151 clerks (83.9% of the target population) and 260 residents (74% of the target population) from 31 postgraduate programs. The factor analyses showed an eight factor instrument. ACLEEM was found highly reliable with a Cronbach's alpha of 0.94 and D-study revealed a reliable outcome for residency programs with at least 15 respondents with a G coefficient of 0.831. The EE perceived by residents and clerks was positive without differences between groups: 152.52 ± 23.36 (76.26%) and 150.61 ± 24.62 (75.30%), respectively (p = 0.761). CONCLUSIONS: The 50-item ACLEEM inventory is a multidimensional and valid instrument requiring only 15 respondents for reliable RESULTS: We recommend using it to measure the EE in the ambulatory postgraduate Spanish-speaking programs

INTRODUCCIÓN Y OBJETIVOS: Las percepciones de los estudiantes sobre el ambiente educacional (AE) han sido estudiadas en programas de pregrado y posgrado. El cuestionario Ambulatory Care Learning Educational Environment Measure (ACLEEM) es un instrumento desarrollado para medir el ambiente educacional en programas de posgrado ambulatorios. El objetivo de este estudio fue evaluar las propiedades psicométricas de este instrumento. MÉTODOS: Se utilizó metodología mixta para desarrollar el instrumento ACLEEM, incluyendo: teoría fundada (8 grupos focales) y panel Delfi de 2 rondas (consenso). Se realizó un estudio piloto y luego se refinó el cuestionario a 50 ítems, que fue administrado prospectivamente en una cohorte de internos y residentes chilenos durante los años 2010-2011. Las evaluaciones psicométricas incluyeron análisis factorial seguido de rotación Varimax (validez de constructo), coeficiente alfa de Cronbach (consistencia interna) y teoría de la generalizabilidad (confiabilidad). RESULTADOS: Se obtuvieron 411 respuestas: 151 internos y 260 residentes (tasa de respuesta: 83,9% y 74%, respectivamente) correspondientes a rotaciones de internado y 31 programas de posgrado en Medicina. El análisis factorial mostró un instrumento de 8 categorías. ACLEEM presentó una alta confiabilidad (alfa de Cronbach 0,94) y un estudio de generalizabilidad que mostró resultados confiables en programas con al menos 15 encuestados (coeficiente G de 0,831). El ambiente educacional percibido por los residentes e internos fue positivo, sin diferencias entre ambos grupos: 152,52 ± 23,36 (76,26%) y 150,61 ± 24,62 (75,30%), respectivamente (p = 0,761). CONCLUSIONES: El cuestionario ACLEEM de 50 ítems es un instrumento multidimensional y válido, que requiere solo 15 encuestados para contar con resultados confiables. Recomendamos su uso para medir el ambiente educacional en programas de posgrado ambulatorios de habla hispana

Identificación de las necesidades de capacitación docente de los jefes de programa de especialización médica / Identifying the main training needs of postgraduate medical program managers

Training of postgraduate medical specialty program managers (PMSPM) is essential for the proper development of their programs. Aim: To identify the main training needs of PMSPM at a medical school. Material and Methods: A mixed-methodology approach was implemented including focus group/interviews and the administration of the Program Managers Training Needs Assessment Questionnaire (PROMANAQ) developed by an expert panel with 59 items (with two sections: relevance/performance-self-perception). Higher priority was assigned to items with high relevance and low performance. Results: Forty five PMSPM completed the PROMANAQ (81.8% response rate). Both sections of PROMANAQ were highly reliable (Cronbach alpha of 0.95/0.97 for relevance/performance-self-perception, respectively). The items with higher priority value were evaluation of clinical educators, evaluation of teaching programs and accreditation of programs. Ten PMSPM were included in the focus group (18.2% of the universe). The findings of the qualitative component were concordant with the areas explored in the questionnaire. Conclusions: The PROMANAQ is valid and reliable to identify the training needs of PMSPM. The views of PMSPM must be taken into account for faculty development planning.

[Identifying the main training needs of postgraduate medical program managers]. / Identificación de las necesidades de capacitación docente de los jefes de programa de especialización médica.

BACKGROUND: Training of postgraduate medical specialty program managers (PMSPM) is essential for the proper development of their programs. AIM: To identify the main training needs of PMSPM at a medical school. MATERIAL AND METHODS: A mixed-methodology approach was implemented including focus group/interviews and the administration of the Program Managers Training Needs Assessment Questionnaire (PROMANAQ) developed by an expert panel with 59 items (with two sections: relevance/performance-self-perception). Higher priority was assigned to items with high relevance and low performance. RESULTS: Forty five PMSPM completed the PROMANAQ (81.8% response rate). Both sections of PROMANAQ were highly reliable (Cronbach alpha of 0.95/0.97 for relevance/performance-self-perception, respectively). The items with higher priority value were evaluation of clinical educators, evaluation of teaching programs and accreditation of programs. Ten PMSPM were included in the focus group (18.2% of the universe). The findings of the qualitative component were concordant with the areas explored in the questionnaire. CONCLUSIONS: The PROMANAQ is valid and reliable to identify the training needs of PMSPM. The views of PMSPM must be taken into account for faculty development planning.

Development of ACLEEM questionnaire, an instrument measuring residents' educational environment in postgraduate ambulatory setting.

BACKGROUND: Students' perceptions of their educational environment (EE) have been studied in undergraduate and postgraduate curricula. Postgraduate EE has been measured in hospital settings. However, there are no instruments available to measure the EE in postgraduate ambulatory settings. AIM: The aim of this study was to develop the "ambulatory care learning education environment measure" (ACLEEM). METHODS: A mixed methodology was used including three stages: (1) Grounded theory (focus groups); (2) Delphi technique to identify consensus; and (3) Pilot study. RESULTS: Three quota samples of approximately 60 stakeholders were formed, one as focus groups and two as Delphi panels. Eight focus groups were carried out including 58 residents (Latin-American Spanish speakers). The results were analysed and 173 items were offered to a National Delphi panel (61 residents and teachers). They reduced in two rounds the number of important items to 54. The 54-item questionnaire was then piloted with 63 residents and refined to the final version of the ACLEEM with 50 items and three domains. CONCLUSIONS: The 50-item inventory is a valid instrument to measure the EE in postgraduate ambulatory setting in Chile. Large-scale administration of the ACLEEM questionnaire to evaluate its construct validity and reliability are the next steps to test the psychometric properties of the instrument.

Evaluación del ambiente educacional en programas de especialización médica / Evaluation of the educational environment in medical specialty programs

Background: The Postgraduate Hospital Education Environment Measure (PHEEM) questionnaire, is a valid and reliable instrument to measure the educational environment (EE) in postgraduate medical education. Aim: To evaluate the EE perceived by the residents of a postgraduate training program using the PHEEM. Material and Methods: The PHEEM was applied in 2010-2011 in 35 specialty programs. We calculated their individual results and compared means of both global and individual domain scores of the PHEEM, by gender, university of origin and nationality. Cronbach's alpha coefficients and D study (Generalizability theory) were performed for reliability. Results: Three hundred eighteen residents were surveyed (75.7% of the total universe). The mean score of the PHEEM was 105.09 ± 22.46 (65.7% of the maximal score) which is considered a positive EE. The instrument is highly reliable (Cronbach's alpha = 0.934). The D study found that 15 subjects are required to obtain reliable results (G coefficient = 0.813). There were no significant differences between gender and university of origin. Foreigners evaluated better the EE than Chileans and racism was not perceived. The programs showed a safe physical environment and teachers with good clinical skills. The negative aspects perceived were a lack of information about working hours, insufficient academic counseling, and scanty time left for extracurricular activities. Conclusions: This questionnaire allowed us to identify positive aspects of the EE, and areas to be improved in the specialty programs. The PHEEM is a useful instrument to evaluate the EE in Spanish-speaking participants of medical specialty programs.

[Evaluation of the educational environment in medical specialty programs]. / Evaluación del ambiente educacional en programas de especialización médica.

BACKGROUND: The Postgraduate Hospital Education Environment Measure (PHEEM) questionnaire, is a valid and reliable instrument to measure the educational environment (EE) in postgraduate medical education. AIM: To evaluate the EE perceived by the residents of a postgraduate training program using the PHEEM. MATERIAL AND METHODS: The PHEEM was applied in 2010-2011 in 35 specialty programs. We calculated their individual results and compared means of both global and individual domain scores of the PHEEM, by gender, university of origin and nationality. Cronbach's alpha coefficients and D study (Generalizability theory) were performed for reliability. RESULTS: Three hundred eighteen residents were surveyed (75.7% of the total universe). The mean score of the PHEEM was 105.09 ± 22.46 (65.7% of the maximal score) which is considered a positive EE. The instrument is highly reliable (Cronbach's alpha = 0.934). The D study found that 15 subjects are required to obtain reliable results (G coefficient = 0.813). There were no significant differences between gender and university of origin. Foreigners evaluated better the EE than Chileans and racism was not perceived. The programs showed a safe physical environment and teachers with good clinical skills. The negative aspects perceived were a lack of information about working hours, insufficient academic counseling, and scanty time left for extracurricular activities. CONCLUSIONS: This questionnaire allowed us to identify positive aspects of the EE, and areas to be improved in the specialty programs. The PHEEM is a useful instrument to evaluate the EE in Spanish-speaking participants of medical specialty programs.